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ABSTRACT 

The paper states that quantification is neither 
ethical nor unethical, but is ethically neutral. It is the behavior 
or intent of the human being that is clearly * matter or: ethical 
concern. Like numerology and the sects of in ame rates and 
qualitatives, there is not so much an unethical practice that is 
supported as there is a lack of vision and concern for other points 
of view. The qualitative and quantitative researchers have made it 
difficult to live in both their worlds even though there are no 
necessary contradictions between them. The quantifiers at the very 
least could have provided data to determine if students ware 
attaining, or were even aware of, aesthetic values and ethical norms. 
If quantifiers detract students from a love of beauty £r*m a 
knowledge of ethical and unethical behavior, then quantifiers are 
failing their responsibilities, and their behaviors are examples of 
ethics gone wrong. Examples are given of scientism, a 
pseudoscientif ic belief that accepts the appearance and apparent 
successes of science without its essence, including a willingness to 
reexamine data, hypotheses, and theories when the evidence so 
suggests. Selected examples are presented regarding educational 
rationalizations that hinder understanding and free inquiry. (PN) 
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THE ETHICS OF QUANTIFICATION AND WHY IT DOESN'T WORK 

O or 

^ Life Among the Numerolog&ts, Inumerates, and Qualitatives 

Gilbert Sax 
University of Washington 

sD Unlike the sheik in a harem who knew what he wanted to do but didn't 

(\| know where to start, I was not only uncertain where I should begin my 

O presentation, but I also had no idea of what it was that frank Besag wanted 

l^j me to do. But taking my cue from an old story about the similarities 

between lawyers and prostitutes (that is, that the results are identical 
although methods may differ), I will assume that the title of my 
presentation was constructed to act as a projective technique that would 
permit me to read into it whatever has lain dormant in my subconscious 
through these many years. The subtitle was added as an extra feature. If 
title and subtitle are unclear now, just wait until my presentation is over 
for a truly polysemous obfuscation. This is not to be confused with 
clarity. - 

I do not believe that quantification is either ethical or unethical 
contrary to the main title of this paper. Guns, bread, and sealing wax are, 
like quantification, ethical neuters. No matter what the gunsmith or baker 
does or plans to do with their products, the products themselves remain 
ethically neutral while the behavior or intent of the human being is clearly 
a matte" of ethical concern. 

I an sure that there are some quantifiers who behave ethically and 
some who do not. The ability to quantify is not emotionally neutral; it has 
either been held in high regard— sometimes bordering on mysticism by the 
numerologists— or it has been vilified by those who believe that the vory 
act of quantification is somehow simplistic, immoral, irrelevant, or 
degrading to what they believe is uniquely human . These vilifiers may be 
referred to as inumerates 1 who are much like their cousins, the illiterates , 
in that neither group is able to understand nor appreciate the range of 
human symbolic language or thought. The inumerates cannot perceive a 
mathematical derivation as beautiful or as something that c*n be uplifting 
to the human spirit. They fail to see the satisfaction that mathematics can 
provide, and like the rest of us, they attack what is not understood. The 
attempt by the extremists to divide research into qualitative and 
quantitativ e types is an example of this attack which was both unnecessary 
and wrong— unnecessary because the topics studied by the "qualitative" 
researcher were also being investigated by the "quantitative" research — 



nT 1 

The term inum jrate was, I believe, coined by Prof. H. Weaver some twenty 
AO years ago when I was teaching at the University of Hawaii. Herb was 

looking for an analogy in math to the illiterate in reading. I think his 
O search was successful. u.s. ^cmrtment of education 
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and wrong because the distinction between the qualitative and quantitative 
are part of the same enterprise and require no rigid distinctions. 

I find it as distressing to recognize ignorance of the a*ts among some 
otherwise intelligent mathematicians and scientists as I do the failure of 
the inumtrates to disregard science and mathematics* A myopic attachment to 
quantification and the concommitant disregard of other concerns, especially 
in the arts, is a form of scientism, a pseudoscientific belief that accepts 
the appear/ nee and apparent successes of science without its essence— a 
willingness *o reexamine data, hypotheses, and theories when the evidence so 
suggests. 

Like numerology and the sects of the inumerates and qualitatives, 
there is not so much an unethical practice that is supported as there is a 
lacK of vision and concern for other points of view. The qualifiers and 
quantifiers have made it difficult to live in both their worlds even though 
there is no necessary contradictions between them. The quantifiers at the 
very least could have provided data to determine if students were 
attaining— ?r were even aware of— aesthetic values and ethical norms. If 
quantifiers detract students from a love of beauty or from a knowledge 
of ethical and unethical behavior, then quantifiers are failing their 
responsibilities, and their behaviors are examples of ethics gone wrong. 

I recently came across an insidious use of quantification. I received 
an article in the mail from a research journal that I was to critique and 
recommend or deny its publication. The paper's unknown author had no data 
or evidence to support an amorphous series of opinions. The author stated 
that the evidence was subtle and could not be quantified in any statistical 
manner. Thus, concluded the author, the opinions presented would just have 
to be accepted. My answer to that was to recommend against publication. If 
some proposition cannot be quantified, that does not even suggest that some 
other nonquantifiable proposition is correct although of course it could be. 
Still, it is the responsibility of the author to garnish whatever evidence 
there is and to submit it for public scrutiny. The inability or 
undesirability of quantifying some proposition or observation is perfectly 
understandable, but it does not follow that because a subtle argument is, in 
fact, subtle, that evidence for the validity of the proposition is true. The 
inability to quantify proves nothing. The sciences— and particularly 

the behavioral sciences— may have created an illusion of false accuracy and 
complexity. In part, this need has been derived from a desire to be 
recognized and counted as part of the classical or "hard" sciences. This is 
s cientism pure and simple, and its proponents deserve no "credit" for being 
ethical. Scientism has lurked in the shadows of statistics— not so much by 
the efforts of the statisticians as by the failure of practitioners and 
users of statistics to behave ethically. If it were ethical to do so, I 
could recite numerous "research" and "evaluation" reports that persuade me 
to believe that the authors were more concerned ;ith political and economic 
realities than they were with ethical issues. A few examples should 
suffice: 

2 



3 



1. An investigator failed to mention the large initial differences 
between experimental and control groups; 

2. In another study, the only data reported were in favor of the 
experimental group (differences favoring the control group were omitted 
although relevant); 

3. No control group was used even though a comparison was possible 
and necessary; 

4* The experimental group was taught a specific lesson that was 
withheld completely from control groups (the conclusion was largely in favor 
of the experimental group); 

5. Findings were suppressed that could, if disseminated, result in 
loss of training funds; 

6. A page was removed from an individually administered standardized 
achievement test and was replaced with easier r >ms without changing norms* 

It is important to keep in mind that these are only a few of the many 
examples that could be used to demonstrate what is clearly unethical. The 
examples I have enumerated here are not spectacular, they are not uncommon. 
They represent not only a callous disregard and concern about ethical 
matters, but they also exhibit a dangerous tendency to take the ax to 
axiological inquiry. In each instance, the responsible persons knew what 
they were doing and the effect that their behavior would have. In 
retrospect, I would argue that no evaluation was conducted in any of these 
"studies;" instead, justifications were prepared for continuing programs of 
questionnable value. As far as the ethics go (and that isn't very far), the 
practices I have described would be unethical for quantitative as well as 
for qualitative research. 

I have not touched upon certain statistical niceties that have found 
their way into "research" reports and were eventually published. Not all of 
these "errors" should be classified as unethical— reflections of ignorance, 
perhaps, but not dishonesty. We are all subject to honest disagreements, 
and prepublication review procedures try hard to catch the most serious 
ones. A breach of ethics assumes that practitioners (whether of science or 
voodoo) know the differences between ethical and unethical behavior and that 
they chose to disregard the road to righteousness. All professions have 
codes of ethics that may not be violated with impunity if the perpetrator is 
caught with hand in till, and education is no exception. Then how do these 
unethical practices "get by"? I suspect that one reason is the "num- 
bers game." This game is played for high stakes, the conclusions are 
known in advance of the treatment, and statistical gobbledegook is 
sufficiently impressive that readers concern themselves with the conclusions 
of a study and will disregard how the study was conducted. In referring to 
the mystique of quantity , Abraham Kaplan (1964) stated that: 
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In these matters the mystique of quantity is especially 
widespread, as hough a statistical formulation somehow provides 
its own content. The magic of numbers cannot produce cognitive 
rabbits out of truly empty hats. In common with all other 
branches of mathematics, statistics alone is but an instrument for 
transforming data, not for producing them. 2 And when the data have 
been cast in statistical form, they are still data; they have not 
thereby been made into a scientific conclusion. The point to the 
statistical formulation and transformation is to enable us to 
extract all the information that the data contain, so that we can 
bring them to bear on the hypotheses for which they are data. (p. 



The statisticians are not entirely innocent in these matters. Uponrare 
occasion they pretend that theirs is the only road to salvation, and by 
doing so they themselves lend credence to the quantitative mystique. Or 
arrogance may intrude upon the scene by attacks on the well-meaning but 
"fuzzy" thinking attributed conveniently to the arts and to whatever is 
meant by qualitative research . To quote Kaplan (1964) once again: 

The point is that both quality and quantity are misconceived when 
they are taken to be antithetical or even alternative. Quantities 
are of qualities, and a measured quality has just the magnitude 
expressed in its measure, (page 207) 

Every profession has developed modes of rationalizing the behaviors 
and beliefs of its members. Unfortunately, education is not exempted from 
this discussion. These rationalizations protect and guard against open 
inquiry and have, from time to time, even been known to stifle communication 
and dissent. A few examples should suffice. For want of a better term, let 
us refer to these rationalizations as hindrances to understanding. Time 
limitations require selective examples. 

1* I have heard more than one faculty member argue that the very 
selection of a research topic is sufficient to demonstrate experimenter 
bias. Their argument goes something like this: first, the selection of a 
research topic depends on many personal and subjective factors; and, second, 
if there is bias in selecting a topic, that bias is likely to spill over 
into methodology a .d into subjective and biased conclusions. A corollary of 
these propositions is that quantification distorts reality by hiding it 
under the cloak of scientism. 



2 

After tie recent allegations regarding the authenticity of Prof. Cyril 
Burt's data, I am not so sure that Kaplan is entirely correct on this 
matter. It may ue that some statisticians are not only the midwives of 
data, but they may even take an active role in quantitative conception. 



If points one and two in the previous paragraph are accepted, we 8re 
indeed, in a morass of philosophical trouble. We might argue that every 
investigation and study has, at one time or other, been selected by someone 
and that therefore we are all guilty of subjectivity and ultimately of 
unethical behavior. It is the jump from subjectivity to unethical behavior 
that causes my grief. Subjectivity , properly identified and described, has 
its place in scientific thinking. It is as silly to imagine a science 
without subjectivity as it is to imagine qualitative research that bears no 
relationships to number systems and processes. In a sense, the qualitative 
researchers have had to come to grips with attacks from "the other side" 
whereas those of us with a more quantitative bent are still fighting members 
of our own team. 

To accept the premise that subjectivity inevitably leads to bias 
is to court disaster, for every choice— scientific or not— will always end 
up as an epistemological caricature in which we play the role of the 
bumbling fool who runs down one path and onto another without thought, 
Although this kind of tomfoolery is distasteful, it does not represent bias, 
which refers to the wanton disregard of evidence or the creation of 
convenient but untrue propositions. It is not the selection of a topic of 
study that leads to bias but rather that once a topic— any topic— is 
selected, bias too easily interferes with judgments. 

2. Quantification can act as a rationalization for failing to educate, 
and we, the quantifiers, have done little to eliminate this problem. The 
assignment of test or course grades can serve as an example of the 
principle. 

I believe it was B.F. Skinner who said that a Frenchman who asks for 
salt in France expects to get salt; in our French classes, an American who 
asks for salt gets an A but no salt. We all abhor the student who "works 
for grades," but we must have forgotten how the student became that way. 
Students do not work for grades when there are no grades to receive. But 
given sufficient time with grades as an incentive and reinforcer, we can't 
deny our role in this matter. Moreover, I will suggest that our behavior in 
this regard is unethical because we have taught the student to work for 
grades rather than for the love of learning when that was our intent. It is 
also unethical if we could have done otherwise. 

At an elementary laboratory school with which I am familiar, no 
students received letter or numerical grades until they attended the eighth 
grade. Up to that point, one could easily observe students actively 
participating in learning— hands were raised to answer teacher questions, 
students volunteered to work on various projects, students arrived early and 
stayed late on their own volition, and the like. But all of that ended when 
grades were first assigned in the eighth grade. Hands that went up before, 
now remain in catatonic inflexibility; the former volunteers now slump in 
their chairs for fear that they will be called upon. From time to time some 
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students do volunteer but only after they determine the amount of credit 
they can expect to receive. 



The grades themselves are not unethical— only people and their 
behaviors can be described in those terms. So haven't we acted unethically 
by pretending that there is some rational way to divide students among the 
letters A to F when we know perfectly well that we cannot do so? Aren't we 
acting unethically when we state that students must earn a score of, say, 
85% (or say any other value that happens to please your fancy) to pass a 
test? If zero percent has no meaning> and if 100 percent has no meaning, 
then I will argue that 85% or any othpr percentage also has no meaning. 

It is important to understand that I am not advocating the elimination 
of grades; my argument is only never to introduce them. Orce assigned, 
they cannot be eliminated easily. Would you give up your paycheck and 
"work for the love of it"? Once grades have reinforcing properties, you 
remove them at your own peril. Grades, then, become goals, and they got 
that way because we wanted a measure that was simple. That type of 
quantification has led to undesirable behaviors that failed to work as we 
wanted them to. What is unethical is our failure to speaK out against a 
practice that we cannot justify. If my daughter receives a C in some 
subject, how can I help her to improve? I suppose that an A means that she 
needs no help and an F that it is too late for help. So grades are not 
diagnostic, their meaning is unclear, and they will not be eliminated until 
something equally coercive and simple comes along. 

Grades do have one advantage that I should mention: they help us to 
rationalize our failure to teach and students to learn. Suppose a class is 
given 10 spelling words to learn and that Randy learns 6. Randy is given a 
grade, say, of C-. Next week another 10 words are given, and Randy learns 
another 6. So far, Randy has missed 8 words that the teacher will probably 
not require Randy to learn. Everyone (including Randy) believes that the 
teacher's job is finished when grades have been assigned. 

3. We are all familiar with the terms under- and over-achievement . 3 > 4 
In using these terms, two measurements are needed: a measure on some 
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I apologize to R.L. Thorndike whose book The Concepts of Over- and 
Underachievement brought these points home in far better fashion than is 
possible here. 
4 

I have been told by some faculty members that overachievement is 
impossible, unlikely, or "bad for children." Underachievement, however, 
is both possible and likely and "bad for children." Fortunately for the 
children, most are "working up to capacity," a situation I have yet to 
observe among adults. 
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criterion value such as an intelligence test score and a rroasure of 
achievement such as is found on many standardized tests* We administer the 
intelligence test and the achievement measure and determine which is higher* 
A high IQ coupled with a low achievement score defines an underachieves the 
opposite relationship leads to o verachievement . Evidently our goal in 
life is to make sure that students are neither unders nor overs ("too much 
of a good thing, you know? 1 *). Fortunately for us, the correlation between 
IQ and achievement test scores is rather high* This is of unestimable 
value in creating "well-adjusted equals ," And with just a bit more effort, 
we can remove the stain of guilt from us and place it squarely where it 
rightly belongs— on the heads of the children, of course! We perform this 
magic act by making sure that we label the students as overs or unders. It 
is not that W3 have a prediction problem; instead, the children have the 
problem* We could create more overs and unders if we could just reduce the 
correlation between IQ and achievement test scores (that should be easy to 
do should any of you be looking for a consultant). By labelling the children 
as urJers and overs , we remove from ourselves any responsibilities that 
someone might dream up. In mastery learning, the children are masters or 
nonmasters , and we are merely the recorders of that difference; the children 
are failures or nonfailures , and our responsibilities end once we push them 
into the correct bin. Note the beauty inherent ir what we are doing. When 
a child gets into trouble we can provide tfie reason: underachievement, 
overachievement, failure to master kejr ideas, learning disabled, ever- 
egocentric, under-egocentric, etc. Of course parents have to be brought 
into the discussion since they may have problems that affect children. The 
perpetuption of the kinds of myths I have described in the preceding 
paragraphs is clearly unethical (and unfair to boot). 

4. We should have joined forces years ago with the qualitative 
researchers. It would have been a winning combination. On one widely used 
reading test, students were asked to select a synonym for start and vacant . 
That particular test yielded scores in the natural sciences, social 
sciences, mathematics, or "general" category. Where do start and vacant 
belong? Responding correctly to either earned points on the social science 
scale (the reference must have been to vacant lots; I am willing to accept 
any hypothesis regarding start ). Items were also classified by type of 
student error. But on some scales, math for example, there was only one 
item on telling time and one item involving subtraction with decimals. 
Instead of just complaining about the lack of items, the joint quantitative- 
qualitative union would have allowed us to categorize some students as 
having dyslexia, functional cerebral disbalance, or idiopathic cerebral 
dysfunctioning. 

5. Just one last topic. Experimentation and quantification have 
worked together for many years and with mutual respect. But even here we 
can get into ethical problems. The December 1970 issue of Psychology Today 
had a few interesting examples that I doubt would get by the University of 
Washington's Behavioral Sciences Review Committee. In one of these exam- 
ples, male subjects were asked to watch semi-nude pictures of men and were 
told that the degree of their own homosexuality could be discerned by look- 
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ing at a calibrated galvonometer. That "galvonometer" was not attached to 
anything, but the experimenters could move the dial. Need I say more? In 
another experiment, subjects were told that they had broken a machine that 
was needed for the experimenter's MA degree; 56% of the subjets were willing 
to sign a petition to double the tuition rates to help an investigator who 
was identified only as belonging to the Young Democrats. 

I am sure that the establishment of peer-review committees has 
improved the ethical responsibilities of universities and subjects who 
participate in studies. My fear is that perhaps we may have gone too far 
and that we might be cheating society by withholding permission to conduct 
some studies. There is a fine line between peer review and censorship, and 
it is getting more difficult to know when to apply each. Perhaps meetings 
such as the one we are holding today can help. 
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NOTES 

Although not referred to in this presentation, I would like to suggest the 
following as an interesting resource: Phillips, D.C. "On What Scientists 
Know, and How They Know It," Chapter III in Elliot Eisner (Ed.) Learning and 
Teaching the Ways of Knowing . Chicago, Illinois: University of Chicago 
Press, 1985, pp. 37-59. 



The current presentation is an excerpt from a paper written by Gilbert Sax 
and Claudia Krenz that will be published, along with the other presenta- 
tions, in book form. The editor is Frank Besag. This paper is copyrighted 
by Gilbert Sax (1986). It may be reproduced only for nonprofit classroom 
purposes. 



